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Abstract 

In this paper we derive for a controlled stochastic evolution system on 
Hilbert space H a sufficient condition for optimality. Our result is derived 
by using its so-called adjoint backward stochastic evolution equation. 
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>: 

o" 1 Introduction 

CN I Consider a stochastic controlled problem governed by the following stochastic 

^1 evolution equation (SEE): 

dX{t) = {AX{t) + b{X{t), v{t)))dt + a{X{t),v{t))dW(t), t G [0, T], 
: \ X(0) = xq. 

■ This system is driven mainly by A is an unbounded linear operator on a separable 

Hilbert space H, a cylindrical Wiener process W on H. The control variable here 
is denoted by i^(-). Then the control problem is to minimize the cost functional, 
which is given by equation f l3.2p in Section El over a set of admissible controls. 

We shall concentrate in providing a sufficient condition for optimality of this 
optimal control problem, which gives this minimization. For this purpose we shall 
apply the theory of backward stochastic evolution equations (or shortly BSEE) as 
in equation (13. 6 p in Section |3l which together with backward stochastic differential 
equations have become nowadays of great interests in many different fields. For 
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example one can see [9] , [H] , [I2] , [13] and [I5] for the applications of backward 
stochastic differential equations in such optimal control problems. 

Our work will not need studying Hamilton- Jacobi-Bellman equation either 
by using semi-group technique or the technique of viscosity solutions. We refer 
the reader to ^ and some of the related references therein for the semi-group 
technique. 

Let us remark that necessary conditions for optimality of the control and 
its corresponding solution X'^^'^ but for the case when a does not depend on u 
can be found in [0]. This is also the case considered in our earlier work in [4j. So 
the present paper generalize the work in [1]. 

2 Notation 

Let {fl, J-", P) be a complete probability space and denote by Af the collection of P- 
nuU sets of J-". Let {W{t), < t < T} be a cylindrical Wiener process on H with 
its completed natural filtration = o W{s) , 0<s<t,ie H*} V A/", t > 0; 

cf. m- 

For a separable Hilbert space E let L'jr{0, T; E) denote the space of all {J^t, < 
t < T} - progressively measurable processes / with values in E such that 



It is known as in [6j that for / G Ljr(0,T; L2{H)), where L2{H) is the space 
of all Hilbert-Schmidt operators on H, the stochastic integral J f{t)dW{t) can be 
defined as a continuous stochastic process in H. The inner product on L2{H) will 




Thus Ll 



(0, T; E) is a Hilbert space with the norm 




be denoted by by (■, 



3 Results 



Let O be a separable Hilbert space equipped with an inner product and 
let U he a convex subset of O. We say that : [0,T] x Q O is admissible if 
G L^(0,r; O) and z/(t) G U a.e., a.s. The set of admissible controls will be 
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denoted by Uad- Let b : H x O ^ H and a : H x O ^ L2{H) be two continuous 
mappings. Consider the following controlled system: 

/ dX{t) = {AX{t) + biX{t),u{t)))dt + aiX{t), u{t))dW{t), 

I ^(0) = a:o, ^'-'^ 

where z/(-) G Uad represents a control variable. A solution of (13. ip will be denoted 
by X'^^'^ to indicate the presence of the control. 

Let i : H X O ^ W and (p : H ^ W he two measurable mappings such that 
the following cost functional is defined: 

J(//(-)):=E[ r i{X'^-\t),u{t))dt + cl){X''^-\T))], v{-)eUad. (3.2) 
Jo 

For example we can take £ and to satisfy the assumptions given in Theorem l3.3[ 
The optimal control problem of the system (13.11) is to find the value function 
J* and an optimal control G Uad such that 

r := inf{J(z/(-)) : E Uad} = J{y*{-))- (3.3) 

If this happens, the corresponding solution X'^*^''^ is called an optimal solution of 
the stochastic control problem (I3.ip - (I3.3I) and [X'^*^''^ , is called an optimal 

pair. 

Let us now state the following result. 

Theorem 3.1 Assume that A is an unbounded linear operator on H that gener- 
ates a Co-semigroup {S{t), t > 0} on H, and b,a are continuously Frechet dif- 
ferentiable with respect to x and their derivatives b^ , (Tx ore uniformly bounded. 
Then for every z/(-) G Uad there exists a unique mild solution X"^ '' on [0,T] 
to ( 13. ip . That is X'^^ '' is a progressively measurable stochastic process such that 
X(0) = Xo and for all t G [0,T], 

X^( )(t) = S{t)xo+ [ S{t- s)b{X^^-\s),iy{s))ds 

Jo 

+ / Sit- s)a{X''^-\s),u{s))dW{s). (3.4) 
Jo 

The proof of this theorem can be derived in a similar way to those in [71 Chapter 
7] or [ID]. 

From here on we shall assume that A is the generator of a Co-semigroup 
{Sit), t > 0} on H. Its adjoint operator A* : ViA*) C H ^ H is then the 
infinitesimal generator of the adjoint semigroup {S*it), t > 0} of {Sit) ,t > 0}. 
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As it is known that backward stochastic differential equations play an impor- 
tant role in deriving the maximum (or minimum) principle for SDEs, it is natural 
to search for such a role for SEEs like (13. ip . For this purpose, let us first consider 
the Hamiltonian: 

n-. H xO X H X L2{H) R, 

n{x, z/, y, z) := £{x, u) + {b{x, v),y)jj + (cr(x, y),z)^. (3.5) 

Consider the following adjoint BSEE on H: 

- dY< \t) = ( A* V^-^it) + V^niX^^-^t), z/(t), ^^"(t), Z'^(-)(t)) ) dt 

-Z''^-^{t))dW{t), 0<t<T, (3.6) 

yK-)(r) = V(/)(X'^(-)(T)), 

where V(p denotes the gradient of (p, which is defined, by using the directional 
derivative D(f){x){h) of at a point x G if in the direction oi h ^ H, as 
(V0(x),/i>^ = D(/)(x)(/i) i=Mh)). 

A mild solution (or briefly a solution) of (13.61) is a pair {Y, Z) G -^i ^) ^ 

L3r(0, T; L2{H)) such that we have P-a.s. for all t G [0, T] 

= 5*(T-t) V0(x'^(-)(r)) 

+ j S*{s-t)V^H{X'^-\s),v{s),Y''^-\s),Z^^-\s))ds 

T 

S*{s-t)Z''^-\s)dW{s). (3.7) 

The existence of such solutions can be obtained from the following theorem. 

Theorem 3.2 ([3J or [8J) Assume that a, I, cf) are continuously Frechet differ- 
entiable with respect to x, the derivatives bj.,crx,(Ju,ix ore uniformly bounded, and 

\(j)x\L(H,H) <C{1 + \x\h) 

for some constant C > 0. 

Then there exists a unique mild solution (Y'^^'\ Z'^^ '') of the BSEE \3. 0\) . 

Let us now consider the following hypothesis: 
a{X''^\t),v{t)) = S{t) ip{X''^\t), v{t)), V G U^d, V t G [0, T], (3.8) 
for some mapping Lp : H x O ^ H satisfying the same properties as a. 
Now we state our main result. 
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Theorem 3.3 For a given admissible control let X'''*^'^ and (Y'^*^'\ Z'^*^'"') 
be the corresponding equations liS. 1\) and liS. 6\) respectively. Assume that liS. 8\) 
holds. Suppose that 
(i) (j) is convex, 

(a) b,a,i are continuously Frechet differentiate with respect to x, u, (p is contin- 
uously Frechet differentiable with respect to x, the derivatives b^, b^, (Jx^cFyAxi C 
are uniformly bounded, and 

\(t)x\L{K,K) < C(l + \x\k) 

for some constant C > 0, 

(m) n{-,-,Y''*^-\t),Z^'{t)) IS convex for all t e [0,T], F-a.s., and 
(iv) n{X^'^\t),v*{t),Y''''^\t),Z^'^\t))= mi^n{X'''^\t),v,Y''^\t^ 

for a.e. t G [0,r], P-a.s. 

Then {X"" v* {■)) is an optimal pair for the problem 113. ip -f TOI) . 

4 Proofs 

In this section we shah estabhsh the proof of Theorem 13. 3[ We need the following 
two lemmas. 

Lemma 4.1 Let i{ji{t) := u*{t)) - fo(X^( )(t), u{t)), where t G [0, T]. If 

assumption (ii) in Theorem \3.3\ holds, then 

E [ ( y''*^-\t) ,x"*(-)(t) - x''^-\t) )] = e[ [ { ,ij,{t) ) dt ] 

Jo 

-E[ [ { VMX''*^-\t),v*{t),Y''\t),Z''*^\t)) ) dt] 

Jo 

+ E[{ Y^'^-\T) , f S{T-t) [a{X^*^-\t),v\t)) - a{X^(-\t),u{t))]dW{t)) ] 
Jo 

/ S{t- s)[a{X''*'^-\s),p\s))-a{X''^-\s),p{s))]dW{s)) dt]. (4.1) 
Jo 

Proof. Thanks to (ii) and (13.11) we find that ipi G -Z^jr(0, T; H). Multiply equation 
(13. 7p by ''piif), integrate with respect to f G [0,T], take the expectation and use 
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stochastic Fubini's theorem to get 



E[ r {Y^*^-\t),Ut))dt] 
Jo 

= E[ / {S*iT-t)Y''\T),i;^{t))dt] 
Jo 

+ E[ [ { [ S*is-t)V,H{X''*^-\s),u*{s),Y'''^-\s),Z''*^-\s))ds,Mi))dt 
Jo Jt 

= E[(r''*"(T), / S{T-t)Mt)dt)] 
Jo 

+ E[ / {W,niX''*^-\t),u*it),Y''\t),Z'''^-\t)) , f Sit-s)Ms)ds)dt] 
Jo Jo 

= E [ ( r'^*(-)(T) ,x'''^-\t)-x''^-\t) > ] 

+ E [ r {VMX''*^-\t),u*{t),Y''\t),Z''*^-\t)) > dt] 

Jo 

-E [ ( y^*(-)(r) , f s{T-t) [a{x^*^-\t)y{t))-a{x^^-\t),vm dW{t)) ] 

Jo 

-E[ r( v,?^(x^*"(i),^*(t),r^*"(t),^^*"w) , 

Jo 

S{t-s) [a{X'''^-\s),v*{s)) - a{X''^-\s),u{s))] dW{s)) dt]. 



Here the following identity is used: 



= f S{t-s) [h{X''*^-\s),v*{s))-h{X''^-\s),v{s)) ]ds 
Jo 

+ f S{t-s)[a{X'''^-\s),u*{s))-aiX^^-\s),u{s))]dW{s), 
Jo 



for < t < T. Thus the lemma follows. ■ 
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Lemma 4.2 Under 1^3. 8\} and assumption (ii) in Theorem \3.3\ we have 

E [ ( Y'^'^-\T) , r SiT - t) [a{X^'^-\t)y{t)) - a{X<\t), u{t)) ] dW{t)) ] 
Jo 

+ E [ ( V^U{X'^*^-\s),p*{s),Y'^*^-\s),Z'^'^-\s)) , 

/ S{t - s) z/*(s)) - a{X''^-\s),v{s)) ] dW{s)) dt ] 

= E[ r {a{X'^*(-\t),u*{t)) - a{X^^-\t),iy{t)) , Z^^^-)(t))^dt]. (4.2) 

^0 

Proof. From fl3.8p it follows that 

a{X''*^-\t), u*{t)) - a{X''^-\t),v{t)) = S{t) ^{t) a.s., 0<t<T, 

for some ij G L^0,T;L2{H)). So by letting f = in (EZD, multiplying it by 
/q ipi't) dW{s) and then taking the expectation to the resulting equation we get 
dfal). ■ 

We are now ready to establish the proof of Theorem 13. 3[ 

Proof of Theorem 13.31 Let iy{-) be an arbitrary admissible control. From the 
definitions in ( 13. 3 p and (13. 2p we obtain 

J(z/*(-))- JK-)) = E[ r iiX-'*^-\t),u*it))dt + <PiX'''^-\T))] 

Jo 

-E[ / £(X'^"(t),zy(t))rft + 0(X^(-)(T))] 
Jo 

= E[ [ WX^*(-)(f),z/*(t))-^(X^(-)(t),Kt)Mt] 
Jo 

+ E [ 0(X'^*(-)(T)) - cf){X''^-\T)) ]. (4.3) 

But 

£(X'^*(-)(t), u*{t)) - e{X''^-\t),u{t)) = U{X'\t),v*{t), Y''*^-\t), z'^*(-)(t)) 

-n{x''^-\t),u{t), Y'*'^-\t),z''''--\t)) 

- {b{x-^''^\t),iy*{t)) - h{x^^-\t),u(t)) , r'^'(-)(t)> 

- {a{X^'^-^{t),u\t))-a{X'^^-^{t),u{t)),Z'^*^-\t))^ a.s. 
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Therefore fl4.3ll becomes 



j(^*(.))_j(^(.))=E 



nix''\t),u*{t),Y''\t),z'''<-\t)) 



-n{x''^\t),u{t),Y^'{t),z^'{t)) 
- {b{x'^*(-\t),u*{t)) -b{x'^^-\t),u{t)) ,y^-\t)) 

+ E [ 0(X'^*(-)(T)) - 0(X''(-)(r)) ]. (4.4) 
By the convexity assumption on (p in (i) we get 

-(/,(X^(-)(T)) < ( V(/)(X'^*(-nT)) ,X^*(-)(T) -X^(T) ) a.s. (4.5) 
But V(/)(X''*( )(T)) = ^^'( ^(T). Thus 

E[0(X'^*(-)(T)) -0(X^(-)(T))] < E[( y^'^-^T) ,X''*(-)(T) -X'^(-)(r) )]. (4.6) 



From (gl]) and (g^D we find that 

E [ ( y"*(-)(T) ,X'^*(-)(T) -X"(-)(T) > ] = E [ r {Y'*^-\t) ,iJi{t) ) f/t] 
- E [ / ( V.n{X''^^-\t),u*{t),Y''\t),Z''*'^-\t)) ,X''*'^-\t) - X''(-\t) ) rft ] 
+ E[ / {a{X''*^-\t),u*{t))-a{X''^-\t),iy{t)),Z''*^-\t))^dt]. (4.7) 



Denote for t e [0,T], 

suit) 



•H(x^*(t),z/*(t),y'^*(-)(t),^"*"(t)) 
-H(x'^(-)(t), z/(t), y"*(-)(t), z'''(-)(t)) 



and 



V'2(t) = 5H(t) + (-^i(t),F'^*«(t)> 

- {aiX'^'^-\t),iy*it))-a{X-^^-\t),u{t)),Z'^'^-\t))^. 



SufRcient conditions for optimality for SEEs 9 
By applying ( KB and (H?711 we obtain 

j(z.*(-)) - J(K-)) < E [ r ( y'^'^w , Mt) > rft ] 

Jo 

+ E [ ( y^'^-^T) , f S{T-t) [a{X'^'^-^{t),u*{t)) - a{X'^^-\t),u{t))]dW{t)) ] 
Jo 

+ E[{VMX'''^-\s),^*{s),Y''''^-\s),Z'''(-\s)) , 

S{t-s) [a{X^'^-\s),u*{s)) -a{X^^-\s),u{s))]dW{s)) dt] 

+ E[ [ Mt)dt]. (4.8) 
Jo 



Hence 



j(z/*(-)) - J(z/(-)) < E [ / snit) dt ] 

Jo 

-E[ r {a{X'''(\t),iy*{t)) - a{X-^^\t), u{t)) , Z'^*^-\t))/t ] 
Jo 

-E[ [ { V,'H{X''*^-\t),iy*{t),Y''''^-\t),Z'''^-\t)) , X^'^'Xt) - X'^-^t) ) dt] 



T 



+ E [ ( V^'^^T) , S{T~ t) [a{X-^'^\t), u*{t)) - a{X'^^-\t),v{t)) ] dW{t)) ] 
Jo 

+ E [ ( VMX''''^-\s),u*{s),Y^'^-\s),Z'''^-\s)) , 

S{t-s) [cx{X''*'^-\s),u*{s)) - a{X'''^-\s),u{s))] dW{s)) dt]. 
So from Lemma [4.21 this inequality becomes 

JK(-))-JK-))< 

-E [ / ( V,'H{X'''^-\t),u*{t),Y'''^-\t),Z'''^-\t)), X''^-\t)-X''^-\t) ) dt] 
Jo 

+ E [ / 5n{t) dt]. (4.9) 







From the convexity assumption of H in condition (iii) we see that the following 
inequality holds for a.e. t G [0,T], P-a.s. 

snit) < ( v,.-H(x'^*(t),z/*(t),F'^*(-)(t),^"*^'Ht)), x"*(-)(t) -x"(-)(t)> 

+ ( V,niX'''it),u*it),Y'''^-\t), Z'^*(-)(t)) , u*it) - u{t) )^. 
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But the minimum condition (iv) implies 

for a.e. t e [0,T], F e.g. [H]. Consequently, for a.e. t G [0,T], P-a.s., 

6nit) - (v,.H(x'^*(t),z/*(t),y"*()(t),z"*()(t)), -x"()(t)) < 0. 

Now by applying this result in (14. 9 p we deduce finally that J(z/*(-)) < J(z/(-)). 
This completes the proof. ■ 

5 Conclusion 

In Theorem 13.31 we derived sufficient conditions for optimality for the optimal 
control problem, which is governed by the SEE (13. ip when the mapping a satisfies 
condition (13. 8 p . It is needed as realized in the proof of Lemma 14.21 The general 
case, i.e. without this latter condition, will be the subject of our next paper. It 
is in fact a non-trivial generalization and to achieve it one needs to include the 
so-called seconde order BSEEs. 
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